Quality report of the IPBES IAS bibliography

Introduction

This report assesses the following in regards to the provided bibliography named bibliography_metrics.rds:

Remarks: - Group ID is in the json, but not in the csv. The group ID makes it possible to directly jump to the reference in the Zotero Library online.

Bibliography Setup

The bibliography is loaded and the DOIs, ISBNs and ISSNs are extracted. In a second step, the corresponding works are downloaded from [OpenAlex(https://openalex.org)].

Show the code 
#|

if (file.exists(params$fn_ilk_bibliography_metrics)) {
    ilk_bibliography <- readRDS(params$fn_ilk_bibliography_metrics)
} else {
    ilk_bibliography <- bibliography_metrics(
        bibliography_zotero = params$ilk_bibliography_zotero,
        bibliography_name = params$ilk_bibliography_name,
        bibliography_url = params$ilk_bibliography_url,
        mc.cores = params$mc.cores
    )
    saveRDS(ilk_bibliography, params$fn_ilk_bibliography_metrics)
}
Show the code 
#|

if (file.exists(params$fn_bibliography_metrics)) {
    bibliography <- readRDS(params$fn_bibliography_metrics)
} else {
    bibliography <- bibliography_metrics(
        bibliography_zotero = params$bibliography_zotero,
        bibliography_name = params$bibliography_name,
        bibliography_url = params$bibliography_url,
        mc.cores = params$mc.cores
    )
    saveRDS(bibliography, params$fn_bibliography_metrics)
}

Data Quality of the Bibliography

Cleanliness of bibliography

One measure of the cleanliness of a Bibliography is assessed by checking the number of references that have a DOI. The following table gives an overview over some numbers regarding the DOIs, ISBNs and ISSNs in the bibliography.

Entries with DOIs, ISBNs or ISSNs

To identify a reference, the most widely used identifier is the DOI. The following table shows the number of references with a DOI and the number of unique DOIs.

To consider duplicate ISBNs or ISSNs as duplicates entries in the library is not waranted as e.g. differenc chapters of a book can be separate entries in the library and therefore lead toi duplicates.

  • DOIs: 3863 (73.8482126%) - 2 duplicates
  • ISBNs: 453 (8.6599121%) - 57 duplicates
  • ISSNs: 3301 (63.1045689%) - 2118 duplicates

The following DOIs are duplicates in the bibliography. This table should be empty.

Show the code 
# duplicate_isbns <- paste0("https://isbnsearch.org/search?s=", bibliography$dois_bib[duplicated(bibliography$isbns)])
# duplicate_issns <- paste0("", bibliography$dois_bib[duplicated(bibliography$issns)])


data.frame(
    Type = "doi",
    Identifier = sprintf('<a href="https://doi.org/%s" target="_blank">%s</a>', bibliography$dois_bib[duplicated(bibliography$dois_bib)], bibliography$dois_bib[duplicated(bibliography$dois_bib)])
) |>
    knitr::kable(
        caption = "Duplicate DOIs in the Bibliography",
        escape = FALSE
    )
Duplicate DOIs in the Bibliography
Type Identifier
doi 10.1111/emr.12464
doi 10.1111/mam.12277

DOIs in Open Alex

To validate the existence and validity of the DOIs, we check if the DOIs are in the OpenAlex database.

Show the code 
dois_not_in_oa <- unique(bibliography$dois_bib)[!(unique(bibliography$dois_bib) %in% bibliography$dois_works)]

dois_valid <- dois_not_in_oa[IPBES.R::doi_valid(dois_not_in_oa)]

Of the 3861 in the library which have a DOI, 471 (12.1989122%) are in not in OpenAlex.

Show the code 
data.frame(
    Type = "doi",
    Identifier = sprintf('<a href="https://doi.org/%s" target="_blank">%s</a>', dois_not_in_oa, dois_not_in_oa)
) |>
    IPBES.R::table_dt(caption = "The Caption")

Of these 3 are not valid. These are:

Show the code 
data.frame(
    Type = "doi",
    Identifier = sprintf('<a href="https://doi.org/%s" target="_blank">%s</a>', dois_not_in_oa[!(dois_not_in_oa %in% dois_valid)], dois_not_in_oa[!(dois_not_in_oa %in% dois_valid)])
) |>
    knitr::kable(
        caption = "Non Valid DOIs in the Bibliography",
        escape = FALSE
    )
Non Valid DOIs in the Bibliography
Type Identifier
doi annurev.es.23.110192.000431
doi 10.1371/ journal.pone.0111913
doi https://www.sciencedirect.com/science/article/abs/pii/S0169534709002018

TODO Finally we check, if these dois exist but are not ingested into OpanAlex. This is done using the doi.org resolver This is disabled at the moment.

Show the code 
dois_exist <- IPBES.R::doi_exists(
    dois_valid,
    cache_file = file.path(".", "cache", "doi_exist.rds")
)
Show the code 
to_check <- bibliography$dois_bib[!(bibliography$dois_bib %in% dois_works)]

dois_valid <- IPBES.R::doi_valid(bibliography$dois_bib)
dois_openalex <- bibliography$dois_bib %in% dois_works
names(dois_openalex) <- bibliography$dois_bib

dois_exist <- IPBES.R::doi_exists(to_check, cache_file = file.path(".", "cache", "doi_exist.rds"))
dois_not_retracted <- IPBES.R::doi_not_retracted(bibliography$dois_bib, cache_file = file.path(".", "cache", "doi_not_retracted.rds"))

sprintf(
    fmt = paste(
        "Number of references: \t\t %d",
        "Number of DOIs: \t\t %d",
        "Number of Duplicate DOIs: \t %d",
        "Number of DOIs in OpenAlex: \t %d ( %f %)",
        "Number of Existing DOIs: \t %d",
        "Number of Retracted DOIs: \t %d",
        "Percentage of Duplicate DOIs: \t %f",
        sep = "\n"
    ),
    nrow(bibliography),
    sum(!is.na(bibliography$dois_bib)),
    length(bibliography$dois_bib) - length(unique((bibliography$dois_bib))),
    sum(dois_openalex), 100 * sum(dois_openalex) / nrow(bibliography),
    sum(dois_exist),
    sum(!dois_not_retracted),
    ((dois_valid |> unique() |> length()) / length(dois_valid)) |> round(digits = 3) * 100
) |> cat()
Show the code 
oldopts <- options(knitr.kable.NA = "")
data.frame(
    Measure = c(
        "# References",
        "**DOI**",
        "# DOIs",
        "# Duplicate DOIs",
        "# Existing DOIs",
        "# Retracted DOIs",
        "% Duplicate DOIs",
        "**ISBN**",
        "# ISBNs",
        "# Duplicate ISBNs",
        "**ISSN**",
        "# ISSNs",
        "# Duplicate ISSNs"
    ),
    Value = c(
        nrow(bibliography),
        NA,
        sum(!is.na(bibliography$dois_bib)),
        length(bibliography$dois_bib) - length(unique((bibliography$dois_bib))),
        sum(dois_exist),
        sum(!dois_not_retracted),
        ((dois_valid |> unique() |> length()) / length(dois_valid)) |> round(digits = 3) * 100,
        NA,
        sum(!is.na(bibliography$isbns)),
        length(bibliography$isbns) - length(unique((bibliography$isbns))),
        NA,
        sum(!is.na(bibliography$issns)),
        length(bibliography$issns) - length(unique((bibliography$issns)))
    )
) |>
    knitr::kable(
        caption = "Cleanliness of the Bibliography",
    )
options(oldopts)

Contentual and Bibliographic analysis

Publication types

Show the code 
bibliography$bibliography |>
    dplyr::group_by(
        Item.Type
    ) |>
    dplyr::summarize(
        count = n()
    ) |>
    dplyr::arrange(
        desc(count)
    ) |>
    knitr::kable()
Item.Type count
journalArticle 4157
bookSection 305
book 244
report 236
webpage 97
conferencePaper 83
document 67
thesis 15
dataset 12
blogPost 9
preprint 2
magazineArticle 1
newspaperArticle 1
presentation 1
videoRecording 1

Year of Publication

Show the code 
data_bib <- bibliography$bibliography |>
    dplyr::group_by(
        Publication.Year,
        Item.Type
    ) |>
    dplyr::summarize(
        count = n(),
        p = count / sum(count),
    ) |>
    dplyr::group_by(
        Item.Type
    ) |>
    dplyr::arrange(
        Publication.Year
    ) |>
    mutate(
        count_cumsum = cumsum(count),
        p_cumsum = cumsum(p)
    ) |>
    dplyr::rename(
        publication_year = Publication.Year,
        type = Item.Type
    ) |>
    dplyr::mutate(
        type = dplyr::case_match(
            type,
            "journalArticle" ~ "article",
            .default = type
        )
    )
`summarise()` has grouped output by 'Publication.Year'. You can override using
the `.groups` argument.
Show the code 
data_works <- bibliography$works |>
    dplyr::group_by(
        publication_year,
        type
    ) |>
    dplyr::summarize(
        count = n(),
        p = count / sum(count),
    ) |>
    dplyr::group_by(
        type
    ) |>
    dplyr::arrange(
        publication_year
    ) |>
    mutate(
        count_cumsum = cumsum(count),
        p_cumsum = cumsum(p)
    ) |>
    dplyr::rename(
        publication_year = publication_year,
        type = type,
        count_oa = count,
        p_oa = p,
        count_oa_cumsum = count_cumsum,
        p_oa_cumsum = p_cumsum
    )
`summarise()` has grouped output by 'publication_year'. You can override using
the `.groups` argument.
Show the code 
data <- dplyr::full_join(
    x = data_bib,
    y = data_works,
    by = c("publication_year", "type")
)

rm(data_bib, data_works)

data |>
    dplyr::filter(publication_year >= 1950) |>
    ggplot() +
    scale_fill_viridis_d(option = "plasma") +
    geom_line(aes(x = publication_year, y = count_cumsum / 10, colour = type), linetype = "solid") + # Zotero
    geom_line(aes(x = publication_year, y = count_oa_cumsum / 10, colour = type), linetype = "dashed") + # OpenAlex
    scale_x_continuous(
        breaks = seq(1500, 2020, 10)
    ) +
    scale_y_continuous(
        "Proportion of publications",
        sec.axis = sec_axis(~ . * 10, name = "Cumulative number of references") # divide by 10 to scale back the secondary axis
    ) +
    labs(
        title = "Publications over time",
        x = "Year",
        y = "Number of publications"
    ) +
    theme_minimal() +
    theme(axis.text.y.right = element_text(color = "red")) +
    theme(legend.position = "bottom") +
    guides(
        fill = guide_legend(
            title = "Legend"
        )
    )
Warning: Removed 36 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 196 rows containing missing values or values outside the scale range
(`geom_line()`).

Show the code 
rm(data)

Access Status of References

This is checked by using the OpenAlex retrieved works. Therefore it is li=mited to the works that are on OpenAlex. At the moment, only references with a DOI were retrieved from OpenAlex.

Show the code 
bibliography$works |>
    ggplot(
        aes(
            x = publication_year,
            fill = oa_status
        )
    ) +
    geom_bar(
        position = "fill"
    ) +
    scale_fill_manual(values = c("#CD7F32", "red", "gold", "green", "pink")) +
    ggtitle("Publication Year") +
    theme(
        plot.title = element_text(size = 15)
    ) +
    theme(legend.position = "bottom")

50 Most often cited Journals

Show the code 
data <- bibliography$bibliography |>
    dplyr::group_by(
        Publication.Title
    ) |>
    dplyr::summarise(
        count = n()
    ) |>
    dplyr::rename(
        Journal = Publication.Title
    ) |>
    dplyr::filter(
        Journal != ""
    ) |>
    dplyr::slice_max(
        count,
        n = 50
    )

data |>
    ggplot(
        aes(
            x = reorder(Journal, count),
            y = count
        )
    ) +
    geom_bar(
        stat = "identity",
        fill = "steelblue"
    ) +
    coord_flip() +
    labs(
        x = "Journal",
        y = "Count"
    ) +
    ggtitle(
        "Most often cited Journals"
    ) +
    theme(
        plot.title = element_text(size = 15)
    )

This table contains all Journals as specified in the Zotero database.

Show the code 
IPBES.R::table_dt(data, "cited_journals")
Show the code 
rm(data)

TODO Coutries of Institutes of all authors

sapply(bibliography\(works\)author, function(x){x[“institution_country_code”]}) |> unlist() |> table() |> sort()

Show the code 
bibliography$works |>
    dplyr::group_by(
        institution_country_code
    )


top_countries <- sapply(bibliography$works$author, function(x) {
    x["institution_country_code"]
}) |>
    unlist() |>
    table() |>
    sort(decreasing = TRUE) |>
    head(50)

data.frame(
    Country = names(top_countries),
    Count = as.numeric(top_countries)
) |>
    ggplot(
        aes(
            x = reorder(Country, -Count),
            y = Count
        )
    ) +
    geom_bar(
        stat = "identity",
        fill = "steelblue"
    ) +
    coord_flip() +
    labs(
        x = "Country",
        y = "Count"
    ) +
    ggtitle(
        "Countries of Institutes of all Authors"
    ) +
    theme(
        plot.title = element_text(size = 15)
    )

VOSViewer Example

VOSViewer

ILK References in Bibliography

This comparison is at the moment done using the DOIs in the target and the ILK bibliography. Entries which have no DOI can not be compared at the mopment. A comparison could be achieved by using text comparison of the title, but this is not implemenmted yet.

From the 3863 references in the target bibliography, 186 (0.05%) are in the ILK bibliography (3091 references).